ADMM Algorithm for Graphical Lasso with an $\ell_{\infty}$ Element-wise Norm Constraint

We consider the problem of Graphical lasso with an additional l∞ element-wise norm constraint on the precision matrix. This problem has applications in high-dimensional covariance decomposition such as in [Janzamin and Anandkumar, 2012a]. We propose an ADMM algorithm to solve this problem. We also use a continuation strategy on the penalty parameter to have a fast implemenation of the algorithm. 1 Problem The graphical lasso formulation with l∞ element-wise norm constraint is as follows: min Θ∈R,Θ≻0 − log det(Θ) + 〈S,Θ〉+ γ‖Θ− diag(Θ)‖1 s.t. ‖Θ− diag(Θ)‖∞ ≤ λ, (1) where ‖ · ‖1 denotes the l1 norm, and ‖ · ‖∞ denotes the l∞ element-wise norm of a matrix. For a matrix X , ‖X‖∞ = max i,j |Xij |. This formulation first appeared in [Janzamin and Anandkumar, 2012a] in the context of high-dimensional covariance decomposition. We next provide an efficient ADMM algorithm to solve (1). 2 ADMM approach The alternating direction method of multipliers (ADMM) algorithm [Boyd et al., 2011, Eckstein, 2012] is especially suited to solve optimization problems whose objective can be decomposed into the sum of many simple convex functions. By simple, we mean a function whose proximal operator can be computed efficiently. The proximal operator of a function f is given by: Proxf (A, λ) = argmin X 1 2 ‖X −A‖F + λf(X) (2) Consider the following optimization problem: min X,Y f(X) + g(Y ) s.t. X = Y , (3) where we assume that f and g are simple functions. The ADMM algorithm alternatively optimizes the augmented Lagrangian to (3), which is given by: Lρ(X,Y ,Λ) = f(X) + g(Y ) + 〈Λ,X − Y 〉+ ρ 2 ‖X − Y ‖F . (4) The (k + 1)th iteration of ADMM is then given by: